Your support helps us better understand what’s motivating the communities we write for and about. Donate today. For more news and context, sign up for our daily newsletter.

For the second year in a row, The 19th and SurveyMonkey have teamed up on a poll to explore core issues in our coverage. SurveyMonkey surveyed more than 20,000 American adults on politics, abortion, gender-affirming care, guns, the workplace and more. Here’s why we tackled this project again — and how.

Why do a poll?

We talk to a lot of people in the course of our reporting on issues that disproportionately affect women, particularly women of color, and LGBTQ+ people. That reporting is invaluable, but it’s just one tool, and one that doesn’t always give us the widest view. This poll sheds light on what women and LGBTQ+ people across America think about the state of the country — its politics, its politicians and its systems. 

Last year’s poll revealed a number of dynamics that helped inform our coverage: that Americans don’t trust politicians to make abortion policy; that as voters, they were motivated by the economy and preserving democracy; and that moms had a harder time than dads balancing work and life.   

Often as we report on other polls, we run into a lack of data on smaller segments of the American population, such as transgender people. SurveyMonkey’s reach allows us to collect significant insight into the views of the communities we cover who are often overlooked in traditional polling. We want to know how our country’s systems are working: What is motivating people? And where do they run into barriers? Doing our own poll with SurveyMonkey is a chance to get new, more detailed data.

What’s different this year? 

Last year we asked questions about experiences with voting; we removed them this year but intend to ask them again in 2024 and future election years. 

We also made some questions more specific, including several on sexual harassment in the workplace and health care discrimination. While that means we can’t directly compare changes over time, we did get much more robust information that better informs us about the challenges people face. 

We added questions about topics that have gained prominence in our politics — including about gender-affirming care and about gun restrictions, ahead of a related Supreme Court case.

How this poll worked — plus how polls work generally

In any public opinion poll, not everyone in the country is asked for their opinion. Pollsters reach out to a representative sample of people and ask a series of questions. Most often outreach occurs through calling landlines or cell phones, though now some pollsters use online or text message surveys. SurveyMonkey, our partner, leverages the reach of its online survey service — engaging more than 2 million people a day — to select a random sample of respondents and ask if they’re willing to answer more questions. 

For this 19th News/SurveyMonkey poll, 20,191 American adults took the survey. The modeled error estimate — the equivalent of a traditional margin of error for surveys of this type — is plus or minus one percentage point. That means the true value of any number for the total number of respondents is expected to be at most one point more or less than what is reported in this survey, at a 95 percent level of confidence. For results based on subgroups, the error estimate will be higher.
SurveyMonkey goes into more detail here.

But how do you know these answers are accurate?

Even if a pollster is really careful with sampling in the traditional manner, they won’t have an exact replica of the total U.S. population. That’s where weighting comes in. SurveyMonkey, like many other pollsters, compares the population who answered the poll to data collected as a part of the Census’s American Community Survey (ACS) along axes of race, sex, age, education and geography. Then the data is weighted so that it matches the population at large. For example, if a survey had significantly more women respondents than men, weighting ensures that the reported results aren’t skewed due to the sample.
It is difficult to accurately weight along areas where the Census does not collect data, like sexuality or gender identity beyond the binary. This year SurveyMonkey was able to weight our subset of transgender respondents by gender and party affiliation, using parameters from a new KFF/Washington Post survey of transgender adults.

What are the limitations of the survey language?

People describe themselves in a lot of ways; the ACS has more limited options. (As The 19th has previously covered, the Census Bureau only just started explicitly gathering data on LGBTQ+ individuals.) To ensure accurate weighting, certain demographic questions in polls must mirror the ACS exactly. That means the phrasing for several questions on our survey deviated from our editorial guidelines around language, which were created to promote clarity as well as equity. 

When respondents were asked how they identify their gender, they were given only three options: male, female or “not listed/non-conforming.” We have used the term “nonbinary” in our reporting to refer to the population that identified as “not listed/non-conforming,” consistent with the recommendation of the Trans Journalist Association. A followup question asked respondents whether they identified as transgender, but we weren’t able to provide more expansive gender options to explicitly collect more granular data on nonbinary individuals. 

In a similar vein, The 19th uses the term Latinx to describe people of Latin American origin or descent. However, the ACS refers to this category as Hispanic or Latino, which is the phrasing used in our survey questions. (We are reporting the results as “Latinx.”) 

Again, SurveyMonkey’s methods allow us to capture a wide range of perspectives and highlight views from segments of the population in a way that a lot of polls don’t allow. But it’s important to also acknowledge our limitations.

Do we still trust polls?

We trust them to do what they’re supposed to do: give us information about public opinion at a certain point in time. By themselves, they’re not intended to predict what will happen, and there’s always some error inherent in any polling figure. That’s why we share how the survey was conducted, as well as the estimate for the potential error. 

“Polls haven’t failed us. They’re still the best way to capture data that accurately reflects the views of millions of people in just a few simple statistics,” said Laura Wronski, director of research at SurveyMonkey. People have particularly high expectations of precision for election polls, she pointed out — maybe unrealistic ones. 

When they see two polls give slightly different numbers for how many people support a policy, for example, they tend to accept the variation, she said. 

“They wouldn’t be so forgiving if the same were true of an election poll — even though they use the same methodology,” Wronski said. “So it’s really all about knowing what the goal of a poll is, and having enough understanding of statistics to interpret the results correctly.”

The 19th worked with SurveyMonkey to make sure we are doing just that. And in the process of reporting, our journalists have also reached out and followed up with respondents to build on the information in hand.

You can see the full data here and the questionnaire here.

Have more questions on how this poll was done, or how polls work in general? Share them with us. 

Methodology: This SurveyMonkey poll was conducted online from August 24-31, 2023 among a national sample of 20,191 adults. Respondents for this survey were selected from the more than 2 million people who take surveys on the SurveyMonkey platform each day. The modeled error estimate for this survey is plus or minus 1.0 percentage points. Data have been weighted for age, race, sex, education, and geography using the Census Bureau’s American Community Survey to reflect the demographic composition of the United States age 18 and over. In addition, data for transgender respondents have been weighted for political party identification and gender using the KFF/The Washington Post Trans Survey to reflect the demographic composition of that subgroup.